ELECTRIC CAMERA 



BACKGROUND OF THE INVENTION 

The present invention relates to a photography 
related to video cameras, camcorders, digital still cameras 
and others using a solid-state image sensing device, and 
more particularly to an electric camera using a solid-state 
image sensing device with a large number of pixels. 

Electric cameras using solid-state image sensors 
such as CCDs (charge-coupled devices) include a so-called 
video camera or camcorder for taking moving images and a 
so-called digital still camera for taking still images. In 
recent years, video cameras with a still image taking func- 
tion and digital still cameras with a moving image taking 
function have become available. 

In a video camera to photograph moving images, it 
is generally assumed that the video is viewed on a display 
such as television monitor and thus the camera is designed 
to produce output signals conforming to a television system 
such as NTSC and PAL. Therefore, the effective number of 
vertically arranged pixels or picture elements on the image 
sensing device used in such a camera is determined to 
enable television signals to be generated. The NTSC 
system, for example, performs interlaced scanning on two 
fields, each of which has an effective scanning line number 
of about 240 lines (the number of scanning lines actually 
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displayed on the monitor which is equal to the number of 
scanning lines in the vertical blanking period subtracted 
from the total number of scanning lines in each field). To 
realize this, the image sensing device has about 480 pixel 
5 rows as the standard effective number of vertically 

arranged pixels. That is, the signals of two vertically 
adjoining pixels in each field are mixed together inside or 
outside the image sensing device to generate about 240 
scanning lines, and the combinations of pixels to be 

10 cyclically mixed together are changed from one field to 
another to achieve the interlaced scanning. 

Some image sensing devices to take moving images 
according to the NTSC system have an area of pixels for 
image stabilization added to the area of effective pixel 

15 area, thus bringing the effective number of vertically 
arranged pixels to about 480 or more. In this case, an 
area beyond 480th pixels is read out at high speed during 
the vertical blanking period and therefore the signals thus 
read out are not used as effective signals. Therefore, the 

20 video signals can only be generated from those signals 
coming from the area of about 480 vertically arranged 
pixels. When such a camera is used to photograph a still 
image, it is relatively easy to generate a static image 
signal conforming to, for example, JPEG (Joint Photographic 

25 Expert Group) from the signals coming from the Scune pixel 
area that is used to take a moving image. A problem 
remains, however, that the number of vertically arranged 
pixels obtained is limited to around 480, making it 
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impossible to produce more detailed static image signals. 

In a camera having an image sensing device with 
the area of pixels for image stabilization mentioned above, 
a method of alleviating this problem may involve using the 
5 entire area of effective pixels including the area of image 
stabilization pixels in photographing a still image. Even 
when photographing a still image, however, the photographed 
image needs to be monitored for check and, for that 
purpose, it is necessary to generate signals conforming to 
10 the television system from signals read out from all effec- 
tive pixels. 

An example of such a conventional camera has been 
proposed in JP-A-11-187306. In the camera disclosed in 
this publication, signals from all the effective pixels are 

15 read out taking two or more times the field period of the 
television system, stored in a memory means such as a field 
memory, and then subjected to interpolation processing for 
trans format ion into signals conforming to the field cycle 
and horizontal scan cycle of television. 

20 This conventional camera, however, requires a 

large processing circuit, such as field memory, for signal 
conversion. Another drawback is that the image sensing 
device readout cycle is a plurality of times the field 
cycle, degrading the dynamic resolution. Even with the use 

25 of this circuit, the number of pixels obtained as the 

static image signals is limited to the number of effective 
pixels used for moving videos plus the area of image 
stabilization pixels. 
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In a digital still camera designed for taking 
still images, there has been a trend in recent years toward 
an increasing niimber of pixels used on the moving video 
image sensing device in order to obtain higher resolution 
5 static image signals. When taking a moving image or 

monitoring the video, it is necessary to generate signals 
that conform to the television system. The number of 
pixels on such an image sensing device, however, does not 
necessarily match the number of scanning lines of the 

10 television system and therefore some form of conversion 
means is required. 

The conversion means may involve, as in the video 
camera with the area of image stabilization pixels, reading 
out signals from the image sensing device taking a longer 

15 time than the field cycle and interpolating them to 

generate television signals. This method has, in addition 
to the problem described above, a drawback that the readout 
cycle increases as the number of pixels increases, further 
degrading the dynamic resolution. 

20 To mitigate this problem, JP-A-9-270959 discloses 

an apparatus which mixes together or culls the pixel 
signals inside the image sensing device to reduce the 
number of signals to be read and therefore the read cycle. 
Although this apparatus alleviates the problem of the 

25 degraded dynamic resolution, it requires a large processing 
circuit such as field memory to perform time-axis transfor- 
mation to generate signals conforming to the television 
system and the image sensing device itself needs to have a 
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special structure for performing desired mixing and 
culling. 

SUMMARY OF THE INVENTION 

The present invention relates to a photography of 
5 video cameras, camcorders, digital still cameras and others 
using a solid-state image sensing device, and more particu- 
larly to an electric camera using a solid-state image 
sensing device with a large number of pixels. 

The conventional electric cameras, as described 

10 above, have drawbacks that when taking a still picture by 
using a video camera, the number of pixels is not suffi- 
cient and that when taking a moving image with a still 
camera, the associated circuit inevitably increases and the 
dynamic image quality deteriorates. Taking both moving and 

15 static images of satisfactory quality with a single camera 
is difficult to achieve. In addition to solving the above 
problems, to obtain good dynamic picture quality by using 
an image sensing device having a large number of pixels 
intended for still images requires extracting a pixel area 

20 that is used to realize an image stabilizing function. The 
conventional art and cameras do not offer a means to 
accomplish this function. 

An object of the present invention is to provide 
an electric camera that solves these problems and which 

25 uses an image sensing device with a sufficient number of 
pixels for still images and enables taking of highly 
detailed still images and a moving video taking with 
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reduced image quality degradation without increasing 
circuitry such as field memory. It is also an object of 
the present invention to provide an electric camera that 
can also realize the image stabilizing function. 
5 According to one aspect of this invention, the 

electric camera to realize the above objectives has: an 
image sensing device with a light receiving surface having 
N vertically arranged pixels and an arbitrary number of 
pixels arranged horizontally, N being equal to or more than 

10 three times the number of effective scanning lines M of a 
display screen of a television system; a driver to drive 
the image sensing device to vertically mix or cull signal 
charges accumulated in individual pixels of every K pixels 
to produce a number of lines of output signals which 

15 corresponds to the number of effective scanning lines M, K 
being at least one of integers equal to or less than an 
integral part of a quotient of N divided by M; and a signal 
processing unit to generate image signals by using the 
output signals of the image sensing device. 

20 As explained above, since this invention 

eliminates the limit on the number of vertically arranged 
pixels, an electric camera can be provided which enables 
taking of highly detailed still images and a satisfactory 
moving video taking by using an image sensing device with a 

25 large enough pixel number even for still images. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram showing the conf igu- 
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ration of a first embodiment of an electric Ccunera accord- 
ing to the present invention* 

Figure 2 is a schematic diagram showing the 
structure of an image sensing device in the first embodi- 
5 ment of the electric camera of the invention . 

Figure 3 is a drive pulse timing diagram in the 
first embodiment of the electric camera of the invention. 

Figure 4 is a schematic diagram showing a mixing 
operation in the first embodiment of the electric camera of 
10 the invention. 

Figure 5 is a schematic diagram showing a readout 
area in the first embodiment of the electric camera of the 
invention. 

Figure 6 is a schematic diagram showing a mixing 
15 operation in the first embodiment of the electric c£unera of 
the invention. 

Figure 7 is a block diagram showing the configu- 
ration of a second embodiment of an electric camera accord- 
ing to the present invention. 
20 Figure 8 is a schematic diagram showing a mixing 

operation in the second embodiment of the electric camera 
of the invention. 

Figure 9 is a schematic diagram showing a readout 
area in the second embodiment of the electric camera of the 
25 invention. 

Figure 10 is a schematic diagram showing the 
structure of an image sensing device in a third embodiment 
of the electric camera according to the. present invention. 
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Figure 11 is a drive pulse timing diagram in the 
third embodiment of the electric camera of the invention. 

Figure 12 is a schematic diagram showing an 
interpolation operation in the third embodiment of the 
5 electric camera of the invention. 

Figures 13A and 13B are schematic diagrams show- 
ing the arrangement of color filters in the image sensing 
device in a fourth embodiment of the electric camera 
according to the present invention. 

10 DESCRIPTION OF THE EMBODIMENTS 

Now embodiments of the present invention will be 
described by referring to the accompanying drawings. 
Figure 1 is a block diagram showing the configuration of 
one embodiment of an electric camera according to the 

15 invention. 

In Figure 1, reference number 1 represents a 
lens, 2 an aperture, 3 an image sensing device, 4 a drive 
circuit, 5 a gain adjust circuit, 6 an analog-digital (A/D) 
conversion circuit, 7 a signal processing circuity 8 a 

20 vertical interpolation circuit to perform interpolation in 
a vertical direction, 9 a horizontal interpolation circuit 
to perform interpolation in a horizontal direction, 10 a 
recording unit including recording media such as magnetic 
tape, semiconductor memory and optical disk to record a 

25 video signal, 11 a control circuit to control these 

constitutional elements according to the operating state / 
12 an encoder circuit to modulate the video signal into a 
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standard television signal, 13 a digital-analog (D/A) 
conversion circuit, 14 a mode selector switch to change 
over the operation mode between the moving video taking and 
the still image taking, 15 a record button to start or stop 
5 the recording, 16a and 16b gyro sensors to detect vertical 
image<-unst ability and lateral image-unstability, respec- 
tively, and 17 an image-unstability decision circuit to 
determine the image- instability from signals output from 
the gyro sensors. 

10 In the above configuration, light coming from the 

lens 1 through the aperture 2 is focused on a light receiv- 
ing surface of the image sensing device 3 where it is 
converted into an electric signal. In this embodiment the 
image sensing device 3 is of a CCD type. Figure 2 shows 

15 the structure of this image sensing device 3. In Figure 2, 
denoted 30 are pixels each formed of a photodiode, which 
are arranged horizontally and vertically in a grid pattern. 
On these grid-arrayed pixels three types of color filters 
that pass yellow (Ye), green (G) and cyan (Cy), respec- 

20 tively, are arranged in such a way that the combination of 
these three colors is repeated horizontally every three 
pixels and that the filters of the same colors are lined 
vertically in so-called vertical stripes. Although an 
arbitrary number of pixels may be used, this embodiment has 

25 an array of 1200 pixels vertically and 1600 pixels horizon- 
tally. A vertical transfer unit 32 is a CCD which is 
driven by three phase pulses VI, V2, V3 . This CCD has a 
three-gate structure in which each pixel corresponds to 
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three phase pulses and thus can vertically transfer a 
signal charge of each pixel independently. Transfer gates 
31 for transferring the charge of each pixel to the 
vertical transfer unit 32 are commonly connected to a gate 
5 of the vertical transfer unit 32 that corresponds to the V2 
pulse. An operation to transfer the charge from each pixel 
to the vertical transfer unit 32 in response to a peak 
value of the pulse applied to the commonly connected gate 
and an operation to transfer the charge vertically are 

10 performed separately. A horizontal transfer unit 33 
horizontally transfers the charges supplied from the 
vertical transfer units 32 and outputs them successively 
through an output amplifier 34 from the output terminal. 
Referring back to Figure 1, the operation 

15 performed when the moving video mode is selected by the 
mode selector switch 14 will be explained. The number of 
vertically arranged pixels on the image sensing device in 
this embodiment is 1200, so if the number of effective 
scanning lines in the field of the NTSC system is assumed 

20 to be 240 lines, then vertically mixing five pixels (- 1200 
pixel rows/240 scanning lines) can match the number of 
lines of output signals from the image sensing device to 
the number of effective scanning lines. 

However/ in this embodiment, to realize the image 

25 stabilizing function described later, four vertically 
arranged pixels are mixed together during motion image 
taking mode, when four vertically arrang d pixels are to 
be cyclically mixed together, the signals from the area of 
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960 pixels (= 240 scanning lines X 4 pixels) out of the 
1200 vertically arranged pixels are used as effective 
signals and the remaining 240 pixels (- 1200 (all pixels) - 
960 (effective pixels)) are not used for image forming, 
5 Figure 3 shows the timing of a vertical drive pulse for the 
image sensing device in this operation mode, with VI, V2 
and V3 representing three phase drive pulses applied to 
each gate of the CCD or vertical transfer unit 32. 

In Figure 3, in a period Tl included in the 

10 vertical blanking period, the drive pulse V2 is held high 
to transfer the signal charge accumulated in each pixel to 
under the V2 gate of the vertical CCD. Next, in a period 
T2, while the V2 pulse is still at middle level, the V3 
pulse is raised from low level to middle level; next, while 

15 the V3 pulse is at middle level, the V2 pulse is changed 

from middle level to low level, after which the VI pulse is 
changed from low level to middle level; next, while the VI 
pulse is at middle level, the V3 pulse is changed from 
middle level to low level, after which the V2 pulse is 

20 changed to middle level and finally the VI pulse is changed 
from middle level to low level. With this sequence of 
pulse operations, the signal charges under the .V2 gate for 
one pixel row are transferred and held again under the V2 
gate. 

25 By repeating this series of operations, the 

signal charges for a desired number of pixel rows can be 
transferred. In Figure 3, during a period T3 included in 
the vertical blanking period before the vertical effective 
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scanning period (the vertical scanning period minus the 
vertical blanking period which corresponds to the actually 
displayed image) and during a period T4 included in the 
vertical blanking period after the vertical effective 
5 scanning period, the above transfer operation for one pixel 
row is repeated a total of 240 times to transfer the signal 
charges of the 240 pixel rows not used for image generation 
to the horizontal transfer unit 33 during the vertical 
blanking period. For example, if this transfer operation 

10 is performed 120 times during the period T3 and 120 times 
during the period T4, the signal charges from upper 120 
pixel rows and lower 120 pixel rows on the light receiving 
surface are transferred to the horizontal transfer unit 33 
during the period T3 and period T4 within the vertical 

15 blanking period. During each of the subsequent periods T5 
and T6 in the vertical blanking period, the horizontal 
transfer unit 33 is driven for a predetermined period to 
output the charges transferred to the horizontal transfer 
unit 33 from the output terminal. These charges are not 

20 used as valid signals as they are output during the 
vertical blanking period. 

Next, in the vertical effective scanning period 
of Figure 3, the above one-pixel-row transfer operation is 
performed four times during each horizontal blanking period 

25 to transfer the signal charges of four pixel rows to the 
horizontal transfer unit 33 where they are mixed together. 
Then, during a horizontal effective scanning period (the 
horizontal scanning period minus the horizontal blanking 
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period which corresponds to the actually displayed image), 
the horizontal transfer unit 33 is driven to read out the 
signal charges from the horizontal transfer unit to produce 
an output signal confozming to the television system. If 
5 the above operation is performed on the A field and if, on 
the B field, the number of pixel rows transferred during 
the period T3 is set to 122 rows and that during the period 
T4 is set to 118 rows, then the combination of four pixel 
rows to be cyclically mixed together shifts by two rows 
10 between the two fields, thus allowing the interlaced scan- 
ning to be performed as shown in Figure 4. (Figure 4 shows 
the light receiving surface of the image sensing device and 
its relation to the displayed screen is vertically 
inverted . ) 

15 Let us return to Figure 1. The output signal 

from the image sensing device 3 is adjusted in gain by the 
gain adjust circuit 5 and then converted by the A/D conver- 
sion circuit 6 into a digital signal. The digital signal 
is then processed by the signal processing circuit 7 that 

20 perfonns color signal processing and luminance signal 
processing, such as generation of color signals, gamma 
correction, white balance processing and outline enhance- 
ment. The image sensing device in this embodiment has an 
array of vertical stripes of yellow (Ye), green (G) and 

25 cyan (Cy) color filters, so the color signals for Ye, G and 
Cy are obtained as a series of color points from one line 
of output signals at all times no matter how many pixels 
are vertically combined. From these color signals three 
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primary color signals R, B can be obtained from the 
following calculations. 

R = Ye - G 
B = Cy - G 
5 G = G 

The R, G and B signals undergoes the white 
balance processing and gamma correction processing in the 
signal processing circuit 7 and are then converted into 
color difference signals such as R-Y^ B-Y or U and V. The 

10 luminance signals and the color difference signals are then 
entered through the vertical interpolation circuit 8 into 
the horizontal interpolation circuit 9. In this operation 
state the signals just pass through the vertical interpola- 
tion circuit 8 without being processed. The horizontal 

15 interpolation circuit 9 performs interpolation on the 
signals in the horizontal direction. 

Figure 5 shows the light receiving surface of the 
image sensing device. As described above ^ in the operating 
state of this embodiment, the signals read out during the 

20 vertical effective scanning period correspond to an area 
having 960 of the 1200 vertically arranged pixels and a 
horizontal width of 1600 pixels, as shown shaded at A in 
Figure 5. If the entire light receiving surface of the 
image sensing device has a 4-to-3 (width to height) aspect 

25 ratio, the shaded area A is more laterally elongate than 

this aspect ratio. Hence, if the signals of all horizontal 
pixels of the light rieceiving surface are displayed, for 
example, on an NTSC standard television monitor with the 
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4-to-3 aspect ratio , the image displayed is compressed 
horizontally and looks vertically elongate, compared with 
the original image. It is therefore necessary to output 
during the horizontal effective scanning period only those 
5 signals coming from a pixel area with the horizontal width 
conforming to the aspect ratio of the television system, as 
shown by a shaded area B. When the television system has 
an 4~to-3 aspect ratio, the number of pixels in the 
horizontal width of the shaded area B is 1280 {= 960 

10 (vertical effective pixels) X 4/3). 

Returning back to Figure 1, the horizontal 
interpolation circuit 9 performs interpolation processing 
on the signals from the horizontal 1280 pixels to expand 
the signals so that they can be output over the entire 

15 horizontal effective scanning period. It also performs 
switching among different clocks as required. With the 
above operation, an area having 960 pixels in height and 
1280 pixels in width is demarcated from the light receiving 
surface as signals conforming to the television system. 

20 Then, the luminance signal and the color difference signal 
are encoded by the encoder circuit 12 into television 
signals, which are then converted by the D/A conversion 
circuit 13 into analog signals for output. When the 
recording is specified by the record button 15, the signals 

25 are recorded by the recording unit 10. At this time, the 
signals may be compressed in the MPEG (Moving Picture 
Expert Group) format. 

Next, the image stabilizing operation will be 
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explained. Image-unstiability information obtained by the 
gyro sensors 16a, 16b that detect vertical and horizontal 
image-unstabilities is entered into the image'-unstability 
decision circuit 17^ which checks the received information 
5 for the amount and direction of the image-unst ability and 
converts them into the number of pixels in vertical and 
horizontal directions on the light receiving surface of the 
image sensing device. Based on the converted pixel 
numbers, the position of an extracted area (effective pixel 

10 area) on the light receiving surface is shifted in a 

direction that cancels the Image-unstability • This can 
correct the image-unstability . The positional shifting of 
the extracted area is performed as follows. The shifting 
in the vertical direction can be made by changing the 

15 number of pixel rows transferred during the periods T3 and 
T4 in Figure 3 and the shifting in the horizontal direction 
made by changing the interpolation start position in the 
horizontal interpolation circuit 9. 

ThiB operation during the moving video mode has 

20 been described above. Next, the operation performed when 
the static image mode is selected by the mode selector 
switch 14 will be explained. 

In the static image mode, too, until the record- 
ing is requested by the record button 15, the camera 

25 outputs signals compatible with the television system to 
monitor the angle of view. Unlike the moving video photo- 
graphing, all of the effective pixels on the image sensing 
device are used in this embodiment during the still image 
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. photographing to produce signals with as high a resolution 
as possible. Hence, during the monitoring the television 
signals need to be generated from the signals coming from 
the entire pixel area. 
5 The image sensing device of this embodiment has 

1200 vertically arranged pixels, and the number of lines of 
output signals from the image sensing device can be made to 
match the number of effective scanning lines of NTSC 
system, which is assumed to have 240 scanning lines, by 

10 vertically mixing five pixels {= 1200/240). To make the 

image sensing device operate in this manner, the one-pixel- 
row transfer operation is performed five times during each 
horizontal blanking period in the vertical effective 
scanning period shown in the pulse timing diagram of Figure 

15 3. With this operation, the signal charges of five pixel 
rows can be mixed by the horizontal transfer unit 33. As 
for the transfer operations during the periods T3 and T4 in 
the vertical blanking period, because the interlaced scan- 
ning is carried out, only two pixel rows are transferred 

20 during the period T3 on the B field, with no transfer 

operations performed in other vertical blanking periods (In 
this embodiment, 1200/240 » 5 with no remainder produced, 
so no further transfer is necessary; if, however, a 
remainder occurs, the remaining pixels need only be trans- 

25 f erred during the periods T3 and T4). 

The charges mixed by the horizontal transfer unit 
33 are read out by driving the horizontal transfer unit 33 
during the horizontal effective scanning period. With the . 
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above operations, the signal charges of all pixels on the 
image sensing device can be read out in a manner conforming 
to the television system. The output signal from the image 
sensing device 3 is^ as during the moving image photo- 
5 graphing, adjusted in gain by the gain adjust circuit 5 and 
converted by the A/D conversion circuit 6 into a digital 
signal, which is then subjected to the color signal 
processing and the luminance signal processing in the 
signal processing circuit 7 before being entered into the 

10 vertical interpolation circuit 8. During the static image 
monitoring, the vertical interpolation circuit 8 performs a 
vertical gravity center correction on the received signals. 

Figure 6 shows combinations of pixels to be 
cyclically mixed on the A field and the B field and also 

15 the vertical position of the gravity center of the mixed 

signals. In the interlaced scanning, scanning lines of the 
A field and the B field are located at the centers of 
adjoining scanning lines on other field. Hence, the signal 
samplings in the camera system for the two fields must be 

20 180 degrees out of phase in the vertical direction. In the 
operating state of this embodiment, however, because five 
pixels are mixed together, the gravity centers of the 
output signals for the A field and the B field are deviated 
36 degrees (= 1/2 pixel or 1/10 the line-to-line distance 

25 on the same field) from the ideal sampling phase difference 
of 180 degrees, as shown in Figure 6. To correct this 
requires generating a signal from two adjoining output 
lines by interpolation. For example, if we let an nth 
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output line on a field be Sn and an (n+l)th output line on 
the same field be Sn+1,. then a signal Sn' obtained by 
calculating Sn' = (Sn X 9/10) + (Sn+1 X 1/10) is one whose 
gravity center is shifted by 1/10 output line from the 
5 gravity center of the nth output line toward the (n+l)th 
output line. The above calculation can also be performed 
on the signals for the B field to correct the gravity 
center deviation of sampling. In this embodiment, however, 
to equalize the effects of interpolation of the A field and 

10 the B field, the following calculations are performed to 
correct the nth output line by 1/20 line toward the (n-1) 
line on the A field and by 1/20 line toward the (n+l)th 
line on the B field. 

A field: Sn' = (Sn X 19/20) + (Sn-1 X 1/20) 

15 B field: Sn' = (Sn X 19/20) + (Sn+1 X 1/20) 

While this embodiment performs the interpolation 
based on the calculation of two adjoining lines of output 
signals, a greater number of lines may be used for the 
interpolation processing. The output signal of the 

20 vertical interpolation circuit 8 is supplied to the 

horizontal interpolation circuit 9, which in this operating 
state does nothing but passes the signal. Then, as in the 
case of the moving image photographing, the signal is 
encoded by the encoder circuit 12 into a television signal, 

25 which is then converted by the D/A conversion circuit 13 
into an analog signal for output. As described above, the 
television signals can be generated from all of the pixel 
area of the image sensing device also during the static 



image mode. 

Next, the operation performed when the recording 
is requested by the record button 15 will be described. 
During the monitoring in the static mode, the signals are 
5 mixed together inside the image sensing device to reduce 
the number of signals and thereby generate television 
signals. During recording, however, the mixing processing 
is not performed and all the pixel signals need to be read 
out independently of each other in order to produce high 

10 resolution signals. To realize this, the one-pixel-row 
transfer operation is performed only once during each 
horizontal blanking period in the vertical effective scan- 
ning period shown in the pulse timing diagram of Figure 3. 
This causes only the signal charges for one pixel row to be 

15 transferred into the horizontal transfer unit 33, which is 
then driven to read out the signal charges for one pixel 
row. Repeating this operation the number of times equal to 
the number of vertically arranged pixel rows enables the 
signal charges of all pixel rows to be read out independ- 

20 ently of each other. The transfer operation is not done 
during the periods T3 and T4 in the vertical blanking 
period. 

The signal charges thus read out are adjusted in 
gain by the gain adjust circuit 5 and converted by the A/D 
25 conversion circuit 6 into digital signals, which are then 
subjected to the color signal processing and the luminance 
signal processing in the signal processing circuit 7, after 
which the signals are supplied through the vertical inter- 
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polation circuit 8 and the horizontal interpolation circuit 
9 to the recording unit 10 where they are recorded. At 
this time, no interpolation processing is performed by the 
vertical interpolation circuit 8 or horizontal interpola- 
5 tion circuit 9. The recording unit 10 may compress the 
signals in the JPEG (Joint Photographic Expert Group) 
format, for example. Because during the static image 
recording, no television signal can be generated, an image 
immediately before starting the recording or a single color 

10 image is output as the television signal for monitoring 

purpose. With the above operation, high resolution signals 
obtained from all the pixels of the image sensing device 
can be recorded. Although in this embodiment the recording 
unit is used commonly for the moving video mode and for the 

15 static image mode, dedicated recording units may be 
provided separately for these modes. 

As explained above, since there is no limit on 
the number of vertically arranged pixels in this embodi- 
ment, an image sensing device with a large enough pixel 

20 number even for still images can be used to photograph 
highly detailed still images and satisfactory moving 
images . 

Further, the signal mixing and the vertical 
signal transfer during the vertical blanking period allow 
25 the signals from the image sensing device with a large 
number of pixels to be read out in. a manner that conforms 
to the television system. This in turn can reduce image 
quality degradation and realize the moving image photo-* 
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graphing with an image stabilizing function and the 
monitoring during still image photographing. 

Only the output signals from that horizontal 
segment which virtually corresponds to the television 
5 system's aspect ratio with respect to the vertical segment 
are extracted and output over the entire horizontal 
effective scanning period of the television system. This 
ensures that the output signals obtained conform to the 
television system's aspect ratio regardless of the 

10 extracted vertical segment position. 

Further, the image sensing device is driven in 
such a way as to shift the position of the pixels to be 
cyclically mixed together every display cycle of the 
television system in order to output interlaced signals. 

15 With this arrangement, the interlaced scanning can be 

performed even when an image sensing device with a large 
number of pixels is used. 

Further, the output signals produced by the 
mixing are interpolated so that the gravity centers of the 

20 output signals interlaced every display cycle have a phase 
difference of 180 degrees in the vertical direction. This 
ensures that the interlaced output signals have no devia- 
tion from the ideal 180-degree phase difference during 
interlacing even when an interlace phase deviation would 

25 normally occur, as when odd numbered pixels are mixed 
together. 

In this embodiment, the image sensing device has 
1200 vertically arranged pixels, and four pixels are mixed 
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Although this embodiment described the case of 
NTSC system, the invention can also be applied to other 
television systems, such as PAL standard, with different 
nvimbers of effective scanning lines. 
5 In summary, a variety of constructions 

essentially equal in the working principle to this embodi*- 
ment can be realized by the use of ah image sensing device 
that has an arbitrary number of vertically arranged pixels 
N three or more times the number of effective scanning 

10 lines M of each field of the television system and which 
allows the vertical mixing or culling of that number of 
pixels which is at least one of integers equal to or less 
than the integral part of a quotient of N divided by M, 

Next, another embodiment of the present invention 

15 will be described by referring to Figure 7 showing the 

configuration of the embodiment. The configuration shown 
in Figure 7 differs from that of Figure 1 in that it has a 
view angle change switch 18. In Figure 7 constitutional 
elements identical with those shown in Figure 1 are 

20 assigned like reference numbers and explanations on the 
constitutional elements performing the same operations as 
those in Figure 1 are omitted here. 

The operations of the moving video mode, the 
monitoring during the static image mode and the static 

25 image recording are similar in normal condition to the 

operations of the previous embodiment which was explained 
with reference to the configuration diagreun of Figure 1 . 
An operation performed when during the moving video mode a 
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request to change the angle of view is made by the view 
angle change switch 18 will be described. 

In the normal condition of this embodiment, as 
.explained in the previous embodiment, the mixing of four 
5 vertically arranged pixels, the vertical transfer during 
the vertical blanking period and the horizontal interpola- 
tion processing are performed to extract an area of 960 
pixels in height and 1280 pixels in width from the entire 
pixel area to generate television signals. When a view 

10 angle change (which means a zooming function without image 
quality degradations in the vertical direction) is 
requested by the view angle change switch 18, three 
vertically arranged pixels are mixed together and the 
signals from the excess vertically arranged pixels are read 

15 out during the vertical blanking periods before and after 
the vertical effective scanning period. 

In this embodiment, signals of 480 pixels (= 1200 
- 240X3) or 160 lines of output signals after mixing 
(= 480/3) need to be read out during the vertical blanking 

20 period. This allows the signals of 720 vertically arranged 
pixels to be read out as 240 lines of output signals 
conforming to the television system. To carry out this 
reading requires, in the pulse timing diagram of Figure 3, 
transferring the signals of three pixel rows during each 

25 horizontal blanking period in the vertical effective scan- 
ning period and also transferring a total of 480 pixel rows 
(= 160 lines of output signals) during the T3 and T4 
periods in the vertical blanking period. The combinations 
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of pixels to be cyclically mixed together are changed from 
one field to another to achieve the interlaced scanning. 

The output signals of the image sensing device 3 
are supplied to the gain adjust circuit 5. Because the 
5 signal level produced as a result of the 3-pixel mixing is 
3/4 the signal level of the 4-pixel mixing ^ the again of 
the gain adjust circuit 5 is increased to 4/3 the gain of 
the 4 -pixel mixing to make the 3- and 4 -pixel-mixed input 
signal levels to the subsequent circuit equal. Then^ the 

10 signals are processed by the A/D conversion circuit 6 and 
the signal processing circuit 7 before being supplied to 
the vertical interpolation circuit 8. The combinations of 
pixels to be cyclically mixed on the A field and the B 
field and the vertical positions of the gravity centers of 

15 the mixed signals are shown in Figure 8. As in the static 
image monitoring of the previous embodiment^ the phase 
difference between the two fields is 180 degrees. Because 
the sampling phases of the fiields are deviated from the 
ideal 180-degree phase difference, the vertical position of 

20 the gravity centers are corrected by the vertical inter- 
polation circuit 8. The amount of phase deviation in this 
operating state is 60 degrees (= 1/2 pixel or 1/6 the line- 
to-line distance on the same field). To correct the phase 
deviation evenly on the both fields, the following calcula- 

25 tions should be performed. 

A field: Sn' = (Sn X 11/12) + (Sn-1 X 1/12) 
B field: Sn' = (Sn X 11/12) + (Sn+1 X 1/12) 
As described earlier, the interpolation process-* 
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ing may use three or more output lines. Next, the 
horizontal interpolation circuit 9 horizontally expands the 
signals from that horizontal segment which corresponds to 
the 4-to-3 aspect ratio with respect to the 720 vertically 
5 arranged pixels (i.e./ signals from a horizontal 960-pixel 
segment (=1600 x 720/1200) in this operating state) so that 
the expanded signals can be output during the entire 
horizontal effective scanning period. With the above 
operation, an area of 720 pixels in height and 960 pixels 

10 in width can be extracted from the light receiving surface. 

Next, when the view angle change is requested 
again by the view angle change switch 18, two vertically 
arranged pixels are mixed together, an area of 480 verti- 
cally arranged pixels is read out during the vertical 

15 effective scanning period, and the horizontal interpolation 
circuit 9 expands the signals from the horizontal 640-pixel 
segment and outputs the expanded signals during the entire 
horizontal effective scanning period to extract an area of 
480 pixels in height and 640 pixels in width. (During the 

20 two-pixel mixing, because the interlace phase deviation 
does not occur, the gravity center position correction by 
the vertical interpolation circuit 8 is not performed.) If 
a further view angle change is requested by the view angle 
change switch 18, the operation is restored to a normal 

25 state where four vertically arranged pixels are mixed 
together . 

As a result of the above operation, the area 
extracted from the light receiving surface of the image 
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sensing device can be changed to three different areas: (A) 
960 pixels high by 1280 pixels wide, (B) 720 pixels high by 
960 pixels wide and (C) 4880 pixels high by 640 pixels 
wide. That is, the angle of view can be changed to three 
5 different angles. With the area A produced by the 4-pixel 
mixing taken as a reference, the area B produced by the 
3-pixel mixing can provide an image enlarged by 1.33 times 
and the area C produced by the 2 -pixel mixing can provide 
an image enlarged by two times. It should be noted here 

10 that because the three different areas are chosen by 
changing the number of pixel to be cyclically mixed 
together in order to make the number of the effective 
output lines from the imaging device agree with the number 
of the effective scanning lines of the television system, 

15 the angle of view can be changed while maintaining a good 
image with no image quality degradation in the vertical 
direction, when compared with an ordinary so-called digital 
zoom which generates, effective scanning lines of signals by 
interpolating a small number of output lines. During the 

20 monitoring of a static image, too, it is possible to 

perform the similar operation of changing the angle of view 
by changing the number of pixel rows to be cyclically mixed 
together . 

As described above, in addition to the advantages 
25 provided by the previous embodiment, this embodiment can 
also realize the view angle change with little image 
quality degradation even for still images by using an image 
sensing device with a large number of pixels and changing 
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the number of pixels to be cyclically mixed together. 

Further, because changes in signal level caused 
when the number of pixels to be cyclically mixed is changed 
are absorbed by the gain adjust means, the input signal 
5 level to the subsequent signal processing means can be kept 
constant. 

While in this embodiment, the view angle change 
is performed by the view angle change switch 18, the angle 
of view may be changed continuously by a zoom switch. In 

10 this case, when the magnification factor does not reach the 
value that is obtained by changing the pixel mixing, the 
digital zoom performs the ordinary interpolation process- 
ing. In this embodiment, when the magnification factor is 
1 or more and less than 1.33, the 4-pixel mixing is 

15 performed; for the factor of 1.33 or more and less than 2, 
the 3 -pixel mixing is done; and for the factor of 2 or 
higher, the 2-pixel mixing is carried out. The mixing 
operation may be interlocked with an optical zooming 
mechanism. 

20 Regardless of the number of pixels in the image 

sensing device, the structure of the image sensing device 
or the television system employed, this embodiment, as in 
the previous embodiment, may also use ^an image sensing 
device that has an arbitrary number of vertically arranged 

25 pixels N three or more times the number of effective scan- 
ning lines M of each field and which allows the vertical 
mixing or culling of those numbers of pixels which are at 
least two of integers equal to or less than the integral 
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part of a quotient of N divided by H. The use of this 
image sensing device can form a variety of constructions 
essentially equal in the working principle to this 
embodiment . 

5 Next, a further embodiment of the present 

invention will be described. The overall configuration of 
this embodiment is similar to that of Figure 1/ except that 
the inner structure of the image sensing device 3 is 
different. The configuration of the image sensing device 

10 in this embodiment is shown in Figure 10. In Figure 10, 
denoted 30 are pixels formed of photodiodes, which are 
arranged horizontally and vertically in a grid pattern. On 
these grid-arrayed pixels three types of color filters that 
pass yellow (Ye), green (G) and cyan (Cy), respectively, 

15 are arranged in so-cail^d vertical stripes. 

In this eiobodiment, the image sensing device has 
an array of pixels measuring 864 pixels vertically and 1152 
pixels horizontally. A vertical transfer unit 32 is a CCD 
which is driven by six phase pulses VI, V2, V3, V4, V5, V6. 

20 This CCD has a two-gate structure in which each pixel 

corresponds to two phase pulses and six gates corresponding 
to the six phase pulses are repeated for every three 
pixels. Transfer gates 31 for transferring the signal 
charge of each pixel to the vertical transfer unit 32 are 

25 commonly connected to respective gates of the vertical 
transfer unit 32 corresponding to the pulses VI, V3, V5. 
An operation to transfer the signal charge from each pixel 
to the vertical transfer unit 32 in response to peak values 
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of pulses applied to the commonly connected gates and an 
operation to transfer the charge vertically are performed 
separately. 

A horizontal transfer unit 33 horizontally 
5 transfers the charges supplied from the vertical transfer 
units 32 and outputs them successively through an output 
amplifier 34 from the output terminal. This image sensing 
device^ unlike the one in the previous embodiment, cannot 
vertically transfer all pixels independently of each other, 

10 but can mix together the signal charges of three vertically 
adjoining pixels inside the vertical transfer unit 32 
before transferring them. 

First, the operation performed in this embodiment 
during the moving video mode will be explained. In the 

15 image sensing device of this embodiment the effective 
number of vertically arranged pixels is 864. If three 
pixels are vertically mixed, the signals of the 720 
(=240X3) of the 864 vertically arranged pixels can be used 
as the effective signals and the remaining 144 (= 864 - 

20 720) pixels can be used as the image-unstability correction 
pixel area . 

Figure 11 shows the timings of vertical drive 
pulses for the image sensing device of Figure 10 during 
this operation mode, with VI, V2, V3, V4, V5, V6 represent- 
25 ing the six phase drive pulses applied to the respective 
gates of the CCD or vertical transfer unit 32. In Figure 
11, during a period Tl included in the vertical blanking 
period, the drive pulses VI, V3 and V5 are held high to 
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cause the signal charge of each pixel to be transferred to 
under the VI ^ V3 and V5 gates of the vertical CCD. Then, 
the V2 and V4 pulses are changed from low level to middle 
level to mix the charges of the adjoining three pixels. 
5 After the mixing, the V5 pulse is changed from middle level 
to low level to hold the mixed signal charges under the VI, 
V2, V3, V4 gates. 

Next, a series of operations performed during a 
period T2 (changing the drive pulses from middle level to 

10 low level or from low level to middle level in the order of 
VI, V2, V3, V4, V5 and V6) causes one mixed output line (3 
pixel rows) to be transferred and held again under the VI, 
V2, V3, V4 gate. By repeating this series of operations, a 
desired number of output lines of mixed signal charges can 

15 be transferred. 

In Figure 11, during a period T3 included in the 
vertical blanking period before the vertical effective 
scanning period and during a period T4 included in the 
vertical blanking period after the vertical effective scan- 

20 ning period, the transfer operation for one output line is 
repeated a total of 144 times to transfer 144 output lines 
of signal charges not used for image forming to the 
horizontal transfer unit 33 at high speed during the 
vertical blanking periods. During subsequent periods T5 

25 and T6 in the vertical blanking periods, the horizontal 
transfer unit 33 is driven for predetermined periods to 
output the signal charges transferred to the horizontal 
transfer unit 33 from the output terminal. 
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Next, in the vertical effective scanning period 
of Figure 11, the one-output-signal-line transfer operation 
is performed during each horizontal blanking period. Then, 
during the horizontal effective scanning period, the 
5 horizontal transfer unit 33 is driven to read out the 

signal charges from the horizontal transfer unit 33. With 
this operation the signal charges of three pixels mixed 
together can be read out in a way conforming to the 
television system. As shown in Figure 11, the signals for 

10 the A field are mixed by changing the V2 and V4 pulses to 
middle level after transferring the signals from the pixels 
to the vertical transfer unit 32. The signals for the B 
field, on the other hand, are mixed by changing the V2 and 
V6 pulses to middle level. With this mixing method, the 

15 combinations of pixels to be cyclically mixed together can 
be changed from one field to another, thereby realizing the 
interlaced scanning. The output signals from the image 
sensing device are processed in the similar manner to that 
of the previous embodiment. A vertical interpolation 

20 circuit 8 performs the gravity center correction, as in the 
3 -pixel mixing in the previous embodiment, and a horizontal 
interpolation circuit 9 performs interpolation processing 
to match the aspect ratio with that of the television 
system. 

25 Next, the operation during the monitoring in the 

static image mode will be explained. It is assumed that 
the still image photographing is done by using all effec- 
tive pixels of the image sensing device, as in the previous 



embodiment. The image sensing device of this embodiment 
has 864 vertically arranged pixels and, when 3-pixel mixing 
is done as in the moving video taking, the nximber of output 
lines is 288 (=864/3), which means that these signal lines 
5 cannot be read out in a manner conforming to the television 
system. Hence, during the monitoring in the static image 
mode, vertical 6-pixel mixing is performed. The 6-pixel 
mixing can be achieved by transferring to the horizontal 
transfer unit 33 in each horizontal blanking period two 

10 output lines of signal charges each of which line has been 
generated by vertically mixing three pixels within the 
vertical transfer unit 32. The 6-pixel mixing can reduce 
the number of output lines from the image sensing device 
down to 144 {= 864/6) lines. The output signals of the 

15 image sensing device that were reduced to 144 output lines 
arie interpolated by the vertical interpolation circuit 8 to 
transform the 144 output lines of signals into 240 lines of 
signals, which conform to the television system. To 
generate 240 lines of signals from the 144 lines requires 

20 interpolation processing that generates five lines from 
three lines (144/240 = 3/5). 

Figure 12 shows how the interpolation is 
performed using two adjoining output lines. Let three 
output lines of the image sensing device be n, n+1 and n+2. 

25 The five output lines of signals can be generated from the 
following calculations, 
n' = n 

n' + 1 = n/2 + (n+l)/2 
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n' + 2 = n -f 1 

n' + 3 = (n+l)/2 + (n+2)/2 

n' + 4 = n + 2 

Three or more output lines of signals may be used 
5 for Interpolation processing. With the above operation, 
television signals can be generated by using signals of all 
pixels of the image sensing device also during the monitor- 
ing in the static image mode. 

Next, the operation performed when the recording 

10 is requested by the record button 15 will be explained. In 
the recording process, the mixing processing is not 
performed and signals of all pixels need to be read 
independently of each other in order to obtain high- 
resolution signals. To realize this, the aperture 2 is 

15 first closed and then, during the period T2 in the pulse 

timing diagram of Figure 11, only the VI pulse is raised to 
high level to transfer the signal charge of only the pixel 
adjacent to the VI gate to the vertical transfer unit 32. 
Then, the vertical transfer unit 32 and the horizontal 

20 transfer unit 33 are successively driven to read out the 
signal charges. Similarly, the V3 pulse is raised to high 
level to read the signal charge of the pixel adjacent to 
the V3 gate, followed by raising the V5 pulse to high level 
to read the signal charge of the pixel adjacent to the V5 

25 gate. With the above processing, the signal charges of all 
pixels can be read out independently in three successive 
operations. The signal charges thus read out are recorded 
in the recording unit 10. At this time, they are 
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rearranged properly to reconstruct the pixel arrangement on 
the light receiving surface of the image sensing device. 

As described above, this emboddjnent offers the 
following advantages. If the number of vertically arranged 
5 pixels ^ is not an integral multiple of the number of scan- 
ning lines of the television system, the signals conforming 
to the television system can be generated from the whole 
area of effective pixels by performing the pixel mixing and 
the vertical interpolation. 

10 In this embodiment, as in the previous embodi- 

ment, regardless of the number of pixels in the image 
sensing device, the structure of the image sensing device 
or the television system employed, a variety of construc- 
tions essentially equal in the working principle to this 

15 ^nbodiment can be realized by using an image sensing device 
that has an arbitrary number of vertically arranged pixels 
N three or more times the number of effective scanning 
lines M of each field and which allows the vertical mixing 
or culling of that number of pixels which is greater by at 

20 least one than the integral part of a quotient of N divided 
by M. 

Next, a further embodiment of the present 
invention will be explained. This embodiment differs from 
the previous embodiments in that the image sensing device 
25 have different arrangements of color filters. Figures 13A 
and 13B show arrangements of color filters in this embodi- 
ment. These color filters in both examples are arranged in 
vertical stripes and, regardless of the number of pixels to 
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be vertically mixed or culled, the R, B primary color 
signals can be generated from one line of output signals. 
Figure 13A shows a color filter arrangement on the image 
sensing device in which white filters (W = passing all 
5 colors) are used instead of the green (G) filters used in 
the previous embodiment. The G, B signals can be 
obtained by the following calculations. 
R = W - Cy 
G = Ye + Cy - W 

10 B = W - Ye 

When this color filter arrangement is used, a 
higher sensitivity can be obtained than the color filter 
arrangement of the previous embodiment. Figure 13B show a 
color filter arrangement that uses, in stead of comple- 

15 mentary colors, color filters that pass the primary colors 
R, G, B. This color filter arrangement can directly 
produce the primary color signals, R, G, B and can provide 
a camera with good color purity and good color S/N. 

With the above color filter arrangements, it is 

20 possible to produce color signals corresponding to the 
three kinds of color filters from each line of output 
signals at all times no matter how many pixels are 
vertically mixed or culled. Therefore, color signals 
conforming to the television system can be generated 

25 easily. 



